
REMARKS 

Initially, Applicants note that on the cover page of the above-mentioned Office 
Action Box 13(c) is checked, indicating that none of the certified copies of the priority 
documents have been received. In fact, the certified copies of both priority documents 
were submitted on July 3, 2001 . A copy of the date-stamped postcard receipt 
acknowledging that submission is attached. Accordingly, Applicants respectfully request 
an indication (Box 13(a)) that all certified copies have been received. 

Applicants request favorable reconsideration and allowance of this application 
in view of the foregoing amendments and the following remarks. 

Claims 1-24 are pending in this application, with Claims 1,21, and 24 being 
independent. Claim 5 has been cancelled without prejudice. 

Claims 1, 4, and 6-21 have been amended. Applicants submit that support for 
the amendments can be found in the original disclosure, and therefore no new matter has 
been added. 

Claims 5, 15, and 16 were objected to. Claim 5 has been cancelled and Claims 
15 and 16 have been amended in view of the Examiner's comments. Applicants submit 
that the amendments overcome the objections, and favorable consideration is requested. 

Claims 1-23 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
U.S. Application No. 5,764,786 (Kuwashima, et al.). Applicants respectfully traverse this 
rejection for the reasons discussed below. 

As recited in Claim 1, the present invention includes, inter alia, the features of 
a detection unit adapted to detect a plurality of feature points and their positions in a two- 
dimensional array on an image pickup screen using a picture in a real space captured by a 
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capture unit; a prediction unit adapted to predict the positions of the feature points in the 
two-dimensional array on the image pickup screen based on the position and orientation of 
the capture unit measured by a position and orientation measurement unit; and a correction 
unit adapted to correct a parameter of the measured position and orientation of the capture 
unit based on the positions of the feature points on the image pickup screen of the capture 
unit obtained by the prediction unit, and based on the positions of the feature points 
obtained by the detection unit. By this arrangement, the position and orientation of the 
capture unit can be determined with high precision, and a parameter according to a 
measurement result obtained by a position and orientation measurement unit can be 
corrected using a plurality of feature points in an image pickup screen. Applicants submit 
that the cited art fails to disclose or suggest at least the above-mentioned features. 

Kuwashima discloses calculating a location based on a capture condition 
(zoom) of a camera and a status of the camera head. That patent also discloses detecting an 
object from a capture image and determining the position of the object based upon the 
position of the object on the captured image. In other words, Kuwashima merely discloses 
a method of detecting a position of an object included in a captured image. However, that 
patent fails to disclose or suggest that the locat ion detected based on the jtatusjxphe 
^camera is corrected using ^nhnage. 

Accordingly, Applicants submit that the present invention recited in Claim 1 is 
patentable over the cited art. Claim 21 recites similar features and is believed allowable for 
similar reasons. 
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The dependent claims recite additional features that further distinguish the 
present invention from the cited art. Individual consideration of those claims is 
respectfully requested. 

New Claim 24 recites, among others, the features of generating a correction 
condition, for correcting a viewing transform condition generated based on a measurement 
result, using a plurality of landmarks-detectedi n a captured im age. Applicants submit that 
the art of record similarly fails to disclose or suggest at least that feature, and therefore 
Claim 24 is also believed to be allowable. 

For the foregoing reasons, Applicants submit that this application is in 
condition for allowance. Favorable reconsideration, withdrawal of the objection and 
rejection set forth in the above-mentioned Office Action, and an early Notice of Allowance 
are requested. 

Applicants' undersigned attorney may be reached in our Washington, DC 
office by telephone at (202) 530-1010. All correspondence should continue to be directed 
to our below-listed address. 



Respectfully submitted, 




Attorney for Applicants 
Brian L. Klock 
Registration No. 36,570 



FITZPATRICK, CELLA, HARPER & SCINTO 
30 Rockefeller Plaza 
New York, New York 101 12-3801 
Facsimile: (212)218-2200 

BLK/lmj 
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APPENDIX 

MARKED-UP VERSION SHOWING AMENDMENTS TO CLAIMS 

1. (Amended) A position and orientation determination apparatus which 
identifies a parameter indicating a position and orientation of a capture [means for 
capturing] unit adapted to capture a picture in a real [a] space containing a plurality of 
feature points whose positions are known in a three-dimensional array, comprising: 

a position and orientation measurement [means for measuring] unit adapted 
to measure the position and orientation of the capture [means] unit in a method other than 
using a captured picture; 

a detection [means for detecting] unit adapted to detect the plurality of 
feature points and their positions in a two-dimensional array on an image pickup screen 
using the picture in the real space captured by [said] the capture [means] unit ; 

a prediction [means for predicting] unit adapted to predict the positions of 
the feature points in the two-dimensional array on the image pickup screen based on the 
position and orientation of [said] the capture [means] unit measured by said position and 
orientation measurement [means] unit ; and 

a correction [means for correcting the parameter indicating the position and 
orientation of said capture means] unit adapted to correct a parameter of the measured 
position and orientation of the capture unit based on the positions of the feature points on 
the image pickup screen of [said] the capture [means] unit obtained by said prediction 
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[means] unit , and based on the positions of the feature points obtained by said detection 
[means,] unit 

[wherein said parameter indicating the position and orientation of said 
capture means for capturing the real space is identified by said correction means correcting 
the parameter]. 

3. (Amended) The position and orientation determination apparatus 
according to claim 1, wherein a point originally existing in the real space as the plurality of 
feature points whose three-dimensional positions are known, and whose two-dimensional 
positions can be detected on the image pickup screen by said detection [means] unit . 

4. (Amended) The position and orientation determination apparatus 
according to claim 1, [wherein on the picture in the real space captured by said capture 
means, another image is superimposed and displayed] further comprising: 

an image generation unit adapted to generate an image of a virtual object 
based on the corrected parameter, and to superimpose the image of the virtual object on the 
picture in the real space captured by the capture unit . 

6. (Amended) The position and orientation determination apparatus 
according to claim 1, wherein said position and orientation measurement [means] unit is a 
sensor for measuring the position and orientation of [said] the capture [means] unit . 
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7. (Amended) The position and orientation determination apparatus 
according to claim 6, wherein said sensor is a magnetic sensor for measuring the three- 
dimensional position and orientation of [said] the capture [means] unit . 

8. (Amended) The position and orientation determination apparatus 
according to claim 1, wherein said correction [means] unit corrects the parameter such that 
[said] the capture [means] unit either rotates or translates. 

9. (Amended) The position and orientation determination apparatus 
according to claim 1, wherein said correction [means] unit corrects the parameter of [said] 
the capture [means] unit by combining rotation transform with translation transform. 



according to claim 9, wherein said correction [means] unit corrects the parameter of [said] 
the capture [means] unit by combining rotation transform with translation transform 
alternately and plural times. 

1 1 . (Amended) The position and orientation determination apparatus 
according to claim 9, wherein said correction [means] unit corrects the parameter such that 
[said] the capture [means] unit can rotate, and then re-predicts the two-dimensional 
position of the feature point on the image pickup screen based on the position of the feature 



10. (Amended) The position and orientation determination apparatus 
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point in the real space and the position and orientation of the camera after the correction, 
and [said] the capture [means] unit can translate. 

12. (Amended) The position and orientation determination apparatus 
according to claim 9, wherein said correction [means] unit corrects the parameter such that 
[said] the capture [means] unit can translate, and then re-predicts the two-dimensional 
position of the feature point on the image pickup screen based on the position of the feature 
point in the real space and the position and orientation of the camera after the correction, 
and [said] the capture [means] unit can rotate. 

13. (Amended) The position and orientation determination apparatus 
according to claim 1, wherein said correction [means] unit computes a first average value 
on the image pickup screen of [said] the capture [means] unit using the position of the 
feature point obtained by said prediction [means] unit , and a second average value on the 
image pickup screen of [said] the capture [means] unit of the feature point detected by said 
detection [means] unit , and corrects the parameter such that the first average value matches 
the second average value. 

14. (Amended) The position and orientation determination apparatus 
according to claim 1, wherein said correction [means] unit computes a first average 
weighting value on the image pickup screen of [said] the capture [means] unit using the 
position of the feature point obtained by said prediction [means] unit , and a second average 
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weighting value on the image pickup screen of [said] the capture [means] unit , of the 
feature point detected by said detection [means] unit , and corrects the parameter such that 
the first average weighting value matches the second average weighting value. 

15. (Amended) The position and orientation determination apparatus 
according to claim 13, wherein when [said] the parameter of [said] the capture [means] unit 
is processed plural times alternately by rotation transform and translation transform, said 
correction [means] unit repeats the process until an average value or [a] an average 
weighting value of an error between the position of the feature point corrected by said 
prediction [means] unit and the position of the feature point on the image pickup screen 
[can be considerably small] is equal to or less than a predetermined value or until the error 
cannot be smaller. 

16. (Amended) The position and orientation determination apparatus 
according to claim 14, wherein when [said] the parameter of [said] the capture [means] unit 
is processed plural times alternately by rotation transform and translation transform, said 
correction [means] unit repeats the process until an average value or [a] an average 
weighting value of an error between the position of the feature point corrected by said 
prediction [means] unit and the position of the feature point on the image pickup screen 
[can be considerably small] is equal to or less than a predetermined value or until the error 
cannot be smaller. 
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17. (Amended) The position and orientation determination apparatus 
according to claim 13, wherein when said correction [means] unit corrects the parameter 
such that [said] the capture [means] unit can rotate, the feature point whose position 
predicted by said prediction [means] unit is associated with the feature point detected by 
said detection [means] unit , a rotation axis and a rotation angle with which the position of 
the feature point on the image pickup screen of [said] the capture [means] unit using the 
position of the feature point obtained by said prediction [means] unit matches the position 
of the feature point on the image pickup screen of [said] the capture [means] unit obtained 
by said detection [means] unit are obtained for each feature point, and [said] the parameter 
can be corrected by using average values of the rotation axis and the rotation angle 
obtained for each feature point which is dealt with. 

18. (Amended) The position and orientation determination apparatus 
according to claim 14, wherein when said correction [means] unit corrects the parameter 
such that [said] the capture [means] unit can rotate, the feature point whose position 
predicted by said prediction [means] unit is associated with the feature point detected by 
said detection [means] unit , a rotation axis and a rotation angle with which the position of 
the feature point on the image pickup screen of the capture [means] unit using the position 
of the feature point obtained by said prediction [means] unit matches the position of the 
feature point on the image pickup screen of [said] the capture [means] unit obtained by said 
detection [means] unit are obtained for each feature point which is dealt with, and [said] 
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the parameter can be corrected using average weighting values of the rotation axis and the 
rotation angle obtained for each feature point which is dealt with. 

19. (Amended) The position and orientation determination apparatus 
according to claim 13, wherein when said correction [means] unit corrects the parameter 
such that [said] the capture [means] unit can translate, the feature point whose position 
predicted by said prediction [means] unit is associated with the feature point detected by 
said detection [means] unit , a difference between the position of the feature point obtained 
by said prediction [means] unit and the position of the feature point obtained by said 
detection [means] unit is obtained for each feature point which is dealt with, and [said] the 
parameter can be corrected using average values of the difference obtained for each feature 
point which is dealt with. 

20. (Amended) The position and orientation determination apparatus 
according to claim 14, wherein when said correction [means] unit corrects the parameter 
such that [said] the capture [means] unit can translate, the feature point whose position 
predicted by said prediction [means] unit is associated with the feature point detected by 
said detection [means] unit , a difference between the position of the feature point obtained 
by said prediction [means] unit and the position of the feature point obtained by said 
detection [means] unit is obtained for each feature point which is dealt with, and [said] the 
parameter can be corrected using average weighting values of the difference obtained for 
each feature point which is dealt with. 
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21 . (Amended) A position and orientation determination apparatus which 
identifies a parameter indicating a position and orientation of a capture [means for 
capturing] unit adapted to capture a picture in a real [a] space containing a plurality of 
feature points whose positions are known in a three-dimensional array, comprising: 

a position and orientation measurement [means for measuring] unit adapted 
to measure the position and orientation of the capture [means] unit in a method other than 
using a captured picture; 

a detection [means for detecting] unit adapted to detect the plurality of 
feature points and their positions in a two-dimensional array on an image pickup screen 
using the picture in the real space captured by [said] the capture [means] unit ; 

a prediction [means for predicting] unit adapted to predict the positions of 
the feature points in the two-dimensional array on the image pickup screen based on the 
position and orientation of [said] the capture [means] unit measured by said position and 
orientation measurement [means] unit ; and 

a correction [means for correcting the parameter indicating the position and 
orientation of said capture means] unit adapted to correct a parameter of the measured 
position and orientation of the capture unit based on the positions of the feature points on 
the image pickup screen of [said] the capture [means] unit obtained by said prediction 
[means] unit , and based on the positions of the feature points obtained by said detection 
[means,] unit 
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[wherein said parameter indicating the position and orientation of said 
capture means for capturing the real space is identified by said correction means correcting 
the parameter]. 
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APPENDIX 

MARKED-UP VERSION SHOWING AMENDMENTS TO SPECIFICATION 

The paragraph at page 1, lines 6-20, has been amended as follows. 

-The present invention relates to a position and orientation determination 
apparatus, method, and a storage medium therefor for determining the position and 
orientation of a capture means before performing an image process to generate [a] an 
augmented picture using an image in an unreal space such as computer graphics 
incorporated into the real space by measuring a parameter indicating the position and 
orientation of capture means (image pickup means) for capturing the real space with [the] a 
method other than using a captured picture, and identifying [the] information about a 
picture whose parameter has been captured. The present invention can also be applied to 
any means and method for determining the position and orientation of a capture device 
used for any purpose other than augmenteding the real space and another space image.- 

The paragraph at page 2, lines 6-18, has been amended as follows. 

-Therefore, in the conventional technology, some methods have been 
suggested to improve the precision of an external parameter (position and orientation of a 
camera) for capturing the real space. In one of the methods, a plurality of feature points 
(landmarks) whose positions are known in a three-dimensional array are arranged in the 
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real space. Then, the external parameter of the camera is corrected using an error between 
the actual position of a target landmark, included in some landmarks captured by the 
camera and displayed on the display screen of the camera, and the position of the target 
landmark predicted based on the position and orientation of the camera at the time, 
acquired by a position and orientation sensor.— 

The paragraph starting at page 2, line 23, and ending at page 3, line 4, has 
been amended as follows. 

—In another method, when there are a plurality of landmarks in the real 
space, using three (or one or two) landmarks in the [vision] image captured by a camera, 
and a measurement value measured by a position and orientation sensor, a projective 
matrix from the real space (three-dimensional space) to the display screen (two- 
dimensional plane) of the camera is calculated by a matrix operation, and an external 
parameter of the camera is corrected using the obtained matrix.— 
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